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DETAILED ACTION 

This office action is in response to tine Amendment and Remarl<s filed 1/25/2010, 
in which claims 1-21 were presented for examination. 

Response to Arguments 

Applicant's arguments with respect to claims 1-21 have been considered but are 
moot in view of the new ground(s) of rejection. 

Claim Rejections - 35 USC § 103 

The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

Claims 1-4, and 13-15 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Simmons, US 6,741 ,91 1 , in view of Mcintosh, US 5,1 03,404, in 
further view of Rosenberg, "Virtual fixtures': Perceptual overlays enhance operator 
performance in telepresence tasks" Stanford University, Stanford, CA, August 1994. 

(Claims 1 and 13) Simmons describes a virtual reality encounter system and 
method comprising: a humanoid robot having tactile sensors positioned along the 
exterior of the robot (column 8, lines 39-50), the sensors sending tactile signals to a 
communications network (column 7, lines 29-32); and a body suit having tactile 
actuators (column 6, lines 33-51, column 8, lines 39-50), the actuators receiving the 
tactile signals from the corresponding tactile sensors on the robot from the 
communication network (column 7, lines 29-32), wherein the tactile sensors and the 
corresponding tactile actuators are calibrated in connection with variable sensitivities 
associated with different regions of the human (column 8, line 62- column 9, line 4, 
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column 13, lines 3-28). Further Mcintosh teaches that it was well known in the remote 
robotic control art to calibrate sensors to different levels of sensitivities to overcome the 
problem and allow, according to Mcintosh, individuals to vary the sensitivities of tactile 
feedback to optimize their own degree of sensitivity and control over the manipulator 
(column 1, lines 41-50, column 9, lines 12-52). It would have been obvious to one of 
ordinary skill in the art at the time of the invention to calibrate the tactile actuators of the 
body suit, because as Mcintosh suggests this would allow the user to preset and vary 
the sensitivities of the suit actuators in individual body locations to the user's preferred 
degree of sensitivity of how to feel the remote environment. This would as Simmons 
suggests, allow the user to feel pressure or force that is proportional to the robot at 
remote locations (column 40, lines 39 -45), allowing the user's hands to feel more 
delicate level of sensation or allowing the user to feel superhuman, by feeling only a 
little of a large force (column 40, lines 19-64). 

Simmons further describes the user apparatus overlays supplemental tactile sensations 
from stored virtual tactile sensations that are sent to the body suit (column 8, line 62- 
column 9, line 4, and column 12, lines 56-65) in the form "[user] feels the rocks holding 
him up and the wind shaking his body" in a virtual world that does not exist. Rosenberg 
teaches the use of perceptual overlay in a telepresence system. Rosenberg defines 
perceptual overlay as "abstract sensory information overlaid on top of the reflected 
sensory feedback from a virtual, remote, or real environment... perceptual overlay may 
be composed sensory information ... including visual, auditory, haptic, and tactile 
sensations" (pg 1 1 , section 1 .8, lines 1-7). Further, "perceptual overlays are in fact 



Application/Control Number: 10/734,616 Page 4 

Art Unit: 3661 

virtual constructs, and not truly a part of the distal environment, perceptual overlays can 
be seen as embellishments of the user's perceptual reality" (pg 12, lines 1-2). The 
telepresence system includes a remotely controlled robot having force sensors 
(MERLIN robot arm, located in the "remote environment"); an exoskeleton having force 
actuators (FREFLEX upper body exoskeleton, located in the "operator space" . . . 
"where the human subject is interfaced to the system", figure 5.1 , pg 122 and the tested 
embodiment of a force reflecting joystick, pg 172), a microprocessor located at the 
operator space connected to the user interface exoskeleton and/or joystick (pg 77 and 
tested as lntel-486 computer, section 7.1.1, pg 172); inherently a microprocessor 
comprises a memory; the computer comprising a software package to be executed by 
the computer, the software configured to: retrieve data for generating supplemental 
virtual tactile sensation signals (virtual fixture models computed by MERLIN VME, fed to 
the FREFLEX VME controller, section 5.4.4, pg 133) the processor generated the 
supplemental virtual tactile signals from the corresponding tactile sensors with the 
generated supplemental tactile sensation signals (section 5.4.4.2-5.4.4.5, the fixture 
models were composed of viscous damping calculations, which was used to determine 
the resultant force to be applied to the exoskeleton); and send the overlaid tactile 
signals to the body suit (the calculated force is sent to the FREFLEX VME controller to 
be implemented on the exoskeleton). Rosenberg further teaches that the virtual fixtures 
may be implemented by the user by using a software package, Percept-Station, that 
includes a retrievable computer-program "toolbox" of virtual fixture models (abstract 
perceptual elements), that the user may overlay on the perceived remote environment 
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by using associated keystrokes and menu options (pg 171 , lines 14-33). It would have 
been obvious to a person of ordinary skill in the art at the time of the invention to 
combine the teachings of Rosenberg to the virtual reality system of Simmons in view of 
Mcintosh because as Rosenberg suggests "by overlaying particular configurations of 

perceptual information on top of the reflected sensory feedback from a telepresence 
worksite, operator performance can be greatly enhanced in a variety of manual tasks." 
(Pg 12, lines 8-10.) 

Rosenberg does not describe the operator's computer that overlay the virtual 
fixtures as a "gateway device". However, it was well known In the computer arts that a 
gateway can be embodied in a general computer to perform the protocol conversion or 
the functions that a separate gateway device (such as a router or modem) performs. It 
would have been very obvious to one of ordinary skill in the communication arts to 
utilize a general computer with an Internal gateway device (Internal router or modem) as 
the operator's computer which overlays the virtual fixtures, as this would have been 
merely a design choice of the network system, comprised of well known network 
components. 

(Claims 2 and 14) Simmons further describes motion sensors positioned 
throughout the body suit (column 13, lines 50-67), the motion sensors sending motion 
signals corresponding to movements of each sensor relative to a reference point 
(column 14, lines 30-41), the motion signals transmitted to the communications network 
(column 7, lines 29-32); and the humanoid robot, receiving, from the communications 
network the signals from the motion sensors (column 11, lines 15-60), the signals from 
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the motion sensors causing a movement of the robot that is correlated to a movement of 
the body suit (column 8, lines 23-30). 

(Claims 3 and 15) Simmons further describes wherein the robot includes 
actuators corresponding to the motion sensors, the actuators causing the robot to move 
(figure 2). 

(Claim 4) Simmons describes the robot comprising a body (column 5, 40-58); a 
camera coupled to the body, the camera for sending video signals to the 
communications network (column 7, lines 2-8, column 11, lines 51-52); and suggests 
sending audio information to the local site over the communications networl< (column 
1 1 , lines 51-52) further describing that a sound sensor means is coupled to second 
body (robot) to capture sound for sending audio signals to the communications network 
(column 4, lines 20-28 and claim 49). 

Claim Rejections - 35 USC § 103 

Claims 5, 8, 9, 16 and 18 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Simmons in view of Mcintosh, and Rosenberg as applied to claims 4 
and 14 above, and further in view of Simmons US 20030030397 (Simmons '397). 

(Claims 5, 8, and 16) Simmons describes wherein the user wears a "wrap 
around video display or a holographic display over his eyes" to render the video signals 
received from the camera (column 9, lines 5-10) and a transducer to transduce the 
audio signals received from the sound sensor coupled to the robot (claim 49, column 4, 
lines 20-28). Simmons '397 teaches that the robot would comprise microphones in the 
appropriate ear location to be relative to the ear position of the user (claim 19) and that 
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the head display would comprise of goggles or glasses (claim 1 2). It would have been 
obvious to one of ordinary skill in the art at the time of the invention to combine the 
teachings of Simmons '397 with the invention of Simmons, in view of Mcintosh and 
Rosenberg, because as Simmons '397 suggests, the microphones and the goggles are 
one well known means of creating the 360-degree, stereoscopic, realistic feedback of 
the remote location ([0016]). Simmons further describes wherein the virtual reality 
system comprises an interface having one or more channels configured to receive the 
audio signals form the sound sensors (column 7, lines 29-32); receiving the video 
signals form the camera (column 7, lines 2-8); sending the video signals to the head 
display (column 9, lines 5-10); and sending the audio signals to the sound producing 
means (column 4, lines 20-28). 

(Claims 9 and 18) Simmons further describes wherein the body includes an eye 
socket and the camera is positioned in the eye socket (column 7, lines 2-8). 

Claim Rejections - 35 USC § 103 

Claims 10 and 19 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Simmons in view of Mcintosh, Rosenberg and Simmons '397 as applied to claims 
5 and 16 above, and further in view of Algazi, US 7,333,622. 

Simmons and Simmons '397 describe wherein the sound sensor is positioned on 
the robot relative to the position of the sound receiver on the person and can be refined 
by the shape of the outer ear (column 4, lines 20-28). Algazi teaches it was well known 
in the art to place listening devices in a mannequin having the exact size, shape, and 
acoustic properties of the listener located in the ear canals to replicate the sound 
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signals accurately (column 3, lines 22-52). It would have been obvious to one of 
ordinary skill in the art at the time of the invention to combine the teachings of Algazi 
with the invention of Simmons in view of Mcintosh, Rosenberg and Simmons '397 
because Algazi describes merely one means of achieving the result described by 
Simmons of replicating the 360 degree, precise audio feedback to the remote user. 
Claim Rejections - 35 USC § 103 

Claims 6, 7 and 17 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Simmons in view of Mcintosh, Rosenberg, Simmons '397 and Yee as applied to 
claims 5 and 16 above, and further In view of Abbasi, US 6,786,863. 

Simmons describes a remote robot operating system and method wherein one 
user controls a robot that duplicates the actions of the operator and transmits the 
sensed condition of the robot environment to the operator, wherein the sensed 
environment is overlaid visually and reproduced by actuators and sensors on the 
operator (column 1, line 36-column 2, line 12). Simmons does not describe wherein at 
the location of the operator, a second humanoid robot transmits data to a first location. 
However, Abbasi teaches this duplication of the same system to create an interaction 
between remote users is known. Abbasi teaches a remote physical contact system and 
method wherein a first surrogate (robot) is at a first location, a second surrogate (robot) 
is at a second location, the second surrogate having the same components, actuators, 
and sensors, i.e. a second microphone and second camera (figure 1, elements 35B, 
40B, and 45B); a second display to receive the video signals from a first camera, a 
second earphone to receive audio signals from a first microphone (figure 1 , elements 25 
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and figure 6), and a first gateway device in the first location and a second gateway 
device having a processor in the second location to create the remote interaction via a 
network (computer network 30 between computers 15 and 25, figure 1). It would have 
been obvious to one of ordinary skill in the art at the time of the invention to combine the 
teachings of Abbasi with the invention of Simmons, in view of Mcintosh, Rosenberg and 
Simmons '397, because as Abbasi teaches the use of remote surrogates and expands 
the notion by using dual surrogates for teleconferencing or computer communications, 
adding a capability to engage in all types of physical contact to "provide for the tactile 
sensation so inherent in many forms of human contact." (Column 1, lines 44-64.) 
Further the combination of the prior arts would produce a predictable result by merely 
duplicating the known systems and interchanging the physical locations, as clearly 
suggested by Abbasi. 

Claim Rejections - 35 USC § 103 
Claims 11, 12, 20 and 21 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Simmons in view of Mcintosh, and Rosenberg as applied to claims 5, 
Simmons in view of Mcintosh, Rosenberg, and Algazi as applied to claim 10, and 
Simmons in view of Mcintosh, Rosenberg, and Simmons '397 as applied to claim 16, 
further in view of Yee, US 6,016,385. 

Simmons in view of Simmons '397 describes transmitting video signals to a set of 
goggles but does not specify a receiver or that the data is transmitted wirelessly. 
However, Yee teaches a headset of a user comprises a receiver to receive video 
signals (column 5, lines 1 1-37) and wherein the robot comprises a transmitter to 
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wirelessly send the audio, tactile, motion and video signals to the communications 
network (communications antenna 30). It would have been obvious to one of ordinary 
skill in the art at the time of the invention to combine the invention of Simmons in view of 
Mcintosh, Rosenberg and Simmons '397 with the teachings of Yee because it was well 
known that wireless means would offer the predictable result of more accessible travel 
of the robot and a wider range of motion. 

Conclusion 

Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to CHRISTINE M. BEHNCKE whose telephone number is 
(571)272-8103. The examiner can normally be reached on 8:30 am- 5pm. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Thomas G. Black can be reached on (571) 272-6956. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retheval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

CMB 

/Thomas G. Black/ 

Supervisory Patent Examiner, Art Unit 3661 



